A Risk Assessment Perspective of Current Practice in Characterizing Uncertainties in QSAR Regression Predictions.

The European REACH legislation accepts the use of non-testing methods, such as QSARs, to inform chemical risk assessment. In this paper, we aim to initiate a discussion on the characterization of predictive uncertainty from QSAR regressions. For the purpose of decision making, we discuss applications from the perspective of applying QSARs to support probabilistic risk assessment. Predictive uncertainty is characterized by a wide variety of methods, ranging from pure expert judgement based on variability in experimental data, through data-driven statistical inference, to the use of probabilistic QSAR models. Model uncertainty is dealt with by assessing confidence in predictions and by building consensus models. The characterization of predictive uncertainty would benefit from a probabilistic formulation of QSAR models (e.g. generalized linear models, conditional density estimators or Bayesian models). This would allow predictive uncertainty to be quantified as probability distributions, such as Bayesian predictive posteriors, and likelihood-based methods to address model uncertainty. QSAR regression models with point estimates as output may be turned into a probabilistic framework without any loss of validity from a chemical point of view. A QSAR model for use in probabilistic risk assessment needs to be validated for its ability to make reliable predictions and to quantify associated uncertainty.

Critical assessment of QSAR models of environmental toxicity against Tetrahymena pyriformis: focusing on applicability domain and overfitting by variable selection.

The estimation of the accuracy of predictions is a critical problem in QSAR modeling. The "distance to model" can be defined as a metric that defines the similarity between the training set molecules and the test set compound for the given property in the context of a specific model. It could be expressed in many different ways, e.g., using Tanimoto coefficient, leverage, correlation in space of models, etc. In this paper we have used mixtures of Gaussian distributions as well as statistical tests to evaluate six types of distances to models with respect to their ability to discriminate compounds with small and large prediction errors. The analysis was performed for twelve QSAR models of aqueous toxicity against T. pyriformis obtained with different machine-learning methods and various types of descriptors. The distances to model based on standard deviation of predicted toxicity calculated from the ensemble of models afforded the best results. This distance also successfully discriminated molecules with low and large prediction errors for a mechanism-based model developed using log P and the Maximum Acceptor Superdelocalizability descriptors. Thus, the distance to model metric could also be used to augment mechanistic QSAR models by estimating their prediction errors. Moreover, the accuracy of prediction is mainly determined by the training set data distribution in the chemistry and activity spaces but not by QSAR approaches used to develop the models. We have shown that incorrect validation of a model may result in the wrong estimation of its performance and suggested how this problem could be circumvented. The toxicity of 3182 and 48774 molecules from the EPA High Production Volume (HPV) Challenge Program and EINECS (European chemical Substances Information System), respectively, was predicted, and the accuracy of prediction was estimated. The developed models are available online at http://www.qspr.org site.

Uncertain numbers and uncertainty in the selection of input distributions--consequences for a probabilistic risk assessment of contaminated land.

Risks from exposure to contaminated land are often assessed with the aid of mathematical models. The current probabilistic approach is a considerable improvement on previous deterministic risk assessment practices, in that it attempts to characterize uncertainty and variability. However, some inputs continue to be assigned as precise numbers, while others are characterized as precise probability distributions. Such precision is hard to justify, and we show in this article how rounding errors and distribution assumptions can affect an exposure assessment. The outcome of traditional deterministic point estimates and Monte Carlo simulations were compared to probability bounds analyses. Assigning all scalars as imprecise numbers (intervals prescribed by significant digits) added uncertainty to the deterministic point estimate of about one order of magnitude. Similarly, representing probability distributions as probability boxes added several orders of magnitude to the uncertainty of the probabilistic estimate. This indicates that the size of the uncertainty in such assessments is actually much greater than currently reported. The article suggests that full disclosure of the uncertainty may facilitate decision making in opening up a negotiation window. In the risk analysis process, it is also an ethical obligation to clarify the boundary between the scientific and social domains.